An effective disaster recovery strategy is to replicate important data to remote file-systems using digital communication networks, such as the Internet. This strategy, referred to herein as remote replication, transfers the data, referred to herein as the replication stream, from a local file-system to a remote file-system. This strategy is ineffective when the throughput capacity of the network is smaller than the production rate of the replication stream. The rate of data required for remote replication often exceeds a network's throughput capacity, especially for small to medium sized enterprise customers, with T1 or lower transmission rates being typical. Throughput capacities of available long-haul networks are increasing with new technology at the network, data link, and physical layers. However, these capacities are often still inadequate to meet the needs of a typical replication stream.
Techniques such as compression can increase effective capacity by reducing the needed bandwidth of the replication stream, but the reductions are often insufficient to allow the data to be transmitted faster than the production rate, especially with pre-compressed data.
Another remote replication strategy allows more efficient transmission of file data by using a previous version of a file for comparison. If such a file exists on both the local and remote file-systems, then a description, called a delta, of how to transform the old remote file into the new file can be transferred. Deltas between an old file and a new file are sufficient to constitute a replication stream for the file. This strategy is referred to herein as delta replication.
A generalized strategy of comparing prior and current versions of the same file to find differences can be straightforward and intuitive. But this only works after an initial copy has been transferred. Thus, each time a new file is created, a relatively large amount of information is transferred. A large number of new files can significantly slow the network or even exceed the network capacity.
Reference will now be made to the exemplary embodiments illustrated, and specific language will be used herein to describe the same. It will nevertheless be understood that no limitation of the scope of the invention is thereby intended.